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Amendments to the Claims 

This listing of claims will replace all prior versions, and listings, of claims in the application: 
Listing of Claims: 

1 . (Currently Amended) An audio and video data processor, comprising a non-transitory 
computer readable medium having thereon : 

a selector for selecting from a first audio data stream at least a portion thereof, said first 
audio data stream being synchronized with corresponding visual events in a moving picture 
video data stream; 

an audio feature analyzer for abstracting from said selected portion of said first audio data 
stream a stream of time-varying features representative of time-varying acoustic features and for 
abstracting corresponding time-varying features from an input second audio data stream; 

a timing analysis and waveform editing processor adapted to determine timing 
differences between said stream of time-varying features and said corresponding time-varying 
features and to utilize said timing differences to edit said input second audio data stream to 
produce an edited input second audio data stream from which said timing differences arc 
substantially removed having acoustic features aligned with the acoustic features of said selected 
portion of said first audio data stream : and 

a playback control module adapted to control running of said synchronized first audio 
data and video data streams with said edited input second audio data stream such that said edited 
input second audio data stream replaces said selected portion and is synchronized with said 
corresponding visual events in said video data stream. 

2. (Currently Amended) A data processing system for audio and video data, comprising a 
non-transitory computer readable medium having thereon: 

digitized audio and moving picture video data for providing a first audio data stream 
synchronized with corresponding visual events in a moving picture video data stream; 

timing data representative of a plurality of selected times in a running of said 
synchronized first audio data stream and said video data stream; 
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audio feature data for providing a data stream of time-varying features abstracted from at 
least a selected portion of said first audio data stream and representative of audible time-varying 
acoustic features; 

an audio feature analyzer for abstracting a corresponding stream of time -varying features 
from an input second audio data stream; 

a timing analysis and waveform editing processor adapted to determine timing 
differences between said streams of time -varying features and to utilize said timing differences 
to edit said input second audio data stream and produce edited input audio data from which said 
timing differences arc substantially removed having acoustic features aligned with the acoustic 
features of said selected portion of said first audio data stream ; and 

a playback control module adapted to control running of said synchronized audio data 
and video data streams with said edited input second audio data such that said edited input 
second audio data replaces said selected portion and is synchronized with said corresponding 
visual events in said video data stream. 

3. (Currently Amended) A data processing system according to claim 2, the non-transitory 
computer readable medium further comprising cueing data representative of timing of said 
selected portion of said first audio data stream. 

4. (Currently Amended) A data processing system according to claim 2, the non-transitory 
computer readable medium further comprising additional digitized audio data for providing a 
further audio data stream started concurrently with said video data stream. 



5. (Cancelled) 
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6. (Currently Amended) A method of providing a processing system for audio and video 
data, comprising the steps of: 

storing in a non-transitory computer readable medium d igitized audio and moving picture 
video data for providing a first audio data stream synchronized with corresponding visual events 
in a moving picture video data stream; 

storing in the non-transitory computer readable medium t iming data representative of a 
plurality of selected times in a running of said synchronized audio and video data streams; 

selecting at least a portion of said first audio data stream; 

abstracting from the selected portion of said first audio data stream , using a 
programmable processor executing computer-executable instructions, audio feature data for 
providing a data stream of time-varying features, said audio feature data being representative of 
audible time-varying acoustic features; 

storing the abstracted audio feature data in the non-transitory computer readable medium; 

abstracting a corresponding stream of time-varying features from an input second audio data 
stream; 

storing in the non-transitory computer readable medium a timing analysis and waveform 
editing processor adapted to determine timing differences between said data stream of time- 
varying features and corresponding features abstracted from said input second audio data stream 
and to produce an editing input second audio data stream from which said timing differences arc 
substantially removed having acoustic features aligned with the acoustic features of said selected 

storing in the non-transitory computer readable medium a playback control module for 
controlling running of said synchronized audio data and video data streams with edited second 
input audio data from said processor such that said edited input second audio data replaces said 
selected portion and is synchronized with said corresponding visual events in said moving 
picture video data stream. 



7. (Currently Amended) 



A method according to claim 6, further comprising the step of: 
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storing in the non-transitory computer readable medium c ueing data representative of 
timing of said selected portion of said first audio data stream. 

8. (Currently Amended) A method according to claim 6, further comprising the step of: 

storing in the non-transitory computer readable medium a dditional digitized audio data 
for providing a further audio data stream started concurrently with said video data stream. 

9. (Previously Presented) A method according to claim 6, wherein said timing data further 
comprises gain control data adapted to control audio gain at selected times during a running of 
said synchronized audio and video data stream to effect replacement of said selected portion. 

10. (Currently Amended) A method of processing audio data, comprising the steps of: 

providing an original first audio data stream synchronized with corresponding visual 
events in a moving picture video data stream; 

selecting at least a portion of said original first audio data stream; 

storing in a non-transitory computer readable medium a n input second audio data stream 
substantially in synchronization with a portion of said video data stream corresponding to the 
selected portion of said original audio data stream; 

abstracting from said input second audio data stream a stream of time-varying features of 
the input second audio data stream, said time-varying features being representative of audible 
time- varying acoustic features; 

comparing , using a programmable processor executing computer-executable instructions, 
the abstracted stream of time-varying features from said input second audio data stream with a 
corresponding stream of time-varying features abstracted from said selected portion of said 
original first audio data stream and determining timing differences between said streams of time- 
varying features; 

editing said second audio data stream u tilizing said timing differences to edit said input 




)roduce edited input second audio data from which said timing 
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difforoncos aro substantially removed having acoustic features aligned with the acoustic features 
of said selected portion of said first audio data stream; and 

running said portion of said video data stream with said edited input second audio data 
such that said edited input second audio data replaces said selected portion and is synchronized 
with said corresponding visual events in said moving picture video data stream. 

11. (Cancelled) 

12. (Previously Presented) A method according to claim 1 0, wherein more than one portion of 
said original first audio data stream is selected. 

13. -15. (Cancelled) 

16. (Currently Amended) A non-transitory c omputer readable medium having thereon audio 
and video data processing software comprising: 

a feature analysis program adapted to derive from audio data feature data representative 
of audible time -varying acoustic features of the audio data; 

a comparison and timing program adapted to compare first feature data derived from first 
audio data synchronized with corresponding visual events in moving picture video data with 
second feature data derived from second audio data and to determine timing differences between 
the first and second feature data; 

an editing program adapted to edit the second audio data in dependence upon said timing 
differences such as to provide edited second audio data in a synchronous relation to said first 
audio data; and 

a streaming program adapted to synchronously output said video data and said edited 
second audio data while muting said first audio data whereby said edited second audio data 
replaces said first audio data and is synchronized with said corresponding visual events in said 
video data. 
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17.-29 (Cancelled). 

30. (Currently Amended) A non-transitory c omputer readable medium having thereon 
graphical user interface software comprising: 

a video and graphics display program adapted to control a display screen and an audio 
output to display moving pictures in response to a stream of synchronized video data and audio 
data and to display a plurality of graphically defined control areas on said screen; 

a control module adapted to detect selection of a said control area by coincidence of 
cursor positioning and actuation of a pointing device and to generate respective control signals in 
response to such selection; and 

an output program adapted to respond to said control signals by outputting selected 
sound- and- motion-synchronized streams of moving picture video data and audio data, and said 
output program being further adapted to record an input audio stream supplied thereto during 
said output of synchronized streams while displaying moving pictures in response to the output 
video data stream. 

3 1 . (New) A method of producing an audio signal as a replacement for a pre-recorded audio 
signal synchronized with a pre-recorded video signal, the method comprising the steps of: 

populating a database in a non-transitory computer readable medium with data 
representing the pre-recorded audio signal synchronized with the pre-recorded video signal, and 
data indicative of the timing of the pre-recorded audio signal relative to the pre-recorded video 
signal; 

receiving and storing in the non-transitory computer readable medium an input audio 
signal having timing similar to that of the pre-recorded audio signal; 

providing first feature data that is derived from the pre-recorded audio signal and encodes 
audible time-varying acoustic features of a predetermined type and the timing thereof within the 
pre-recorded audio signal; 
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providing second feature data that is derived from the input audio signal and encodes 
audible time-varying acoustic features of the predetermined type and the timing thereof within 
the input audio signal; 

determining timing differences existing between the first feature data and the second 
feature data; 

altering the timing of the acoustic features of the stored input audio signal to match the 
timing of the corresponding acoustic features of the pre-recorded audio signal to produce thereby 
an edited replacement audio signal; and 

selecting a start time for starting outputting of the edited replacement audio signal to 
coincide with a corresponding start time of the pre-recorded audio signal relative to the pre- 
recorded video signal and for starting muting of the pre-recorded audio signal. 

32. (New) A method according to claim 3 1 , wherein said step of populating a database with 
data includes storing in the database the first feature data, and said step of determining timing 
differences includes retrieving the first feature data from the database. 

33. (New) A method according to claim 3 1 or 32, wherein said step of populating a database 
is performed in a first processing system, and said steps of determining timing differences, and 
altering the timing are performed in a second processing system. 

34. (New) A method according to claim 3 1 , wherein the pre-recorded audio signal is any 
selected one of a plurality of pre-recorded audio signals synchronized with a pre-recorded video 
signal, and said step of determining timing differences includes selecting one of the plurality of 
pre-recorded audio signals. 

35. (New) A method according to claim 3 1 , further comprising outputting the edited 
replacement audio signal while muting the pre-recorded audio signal. 
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36. (New) An apparatus for producing an audio signal as a replacement for a pre-recorded 
audio signal synchronized with a pre-recorded video signal, the apparatus comprising: 
one or more non-transitory computer readable mediums; 

a database populated with data representing the pre-recorded audio signal synchronized 
with the pre-recorded video signal, and data indicative of the timing of the pre-recorded audio 
signal relative to the pre-recorded video signal; 

means for deriving from the pre-recorded audio signal first feature data encoding audible 
time-varying acoustic features of a predetermined type and the timing thereof in the pre-recorded 
audio signal; 

means for receiving and storing an input audio signal having timing similar to that of the 
pre-recorded audio signal; 

means for deriving from the input audio signal second feature data encoding audible time- 
varying acoustic features of said predetermined type and the timing thereof within the input 
audio signal; 

means for determining timing differences existing between the first feature data and the 
second feature data; 

means for altering the timing of the acoustic features of the stored input audio signal to 
match the timing of the corresponding acoustic features of the pre-recorded audio signal and 
producing thereby an edited replacement audio signal; and 

means for selecting a start time for starting outputting of the edited replacement audio 
signal to coincide with a corresponding start time of the pre-recorded audio signal relative to the 
pre-recorded video signal and for starting muting of the pre-recorded audio signal, 

wherein the database, the means for deriving first feature data, the means for receiving 
and storing an input audio signal, the means for deriving second feature data, the means for 
determining timing differences, the means for altering the timing, and the means for selecting the 
start time are each stored in at least one of the one or more non-transitory computer readable 
mediums. 
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37. (New) An apparatus according to claim 36, further including means for populating the 
database with said first feature data. 

38. (New) An apparatus according to claim 36, further including means for populating the 
database with data from the pre-recorded audio signal synchronized with the pre-recorded video 
signal. 

39. (New) An apparatus according to claim 37 or 38, wherein the means for populating the 
database are located in a first processing system and the determining means and altering means 
are located in a second processing system. 

40. (New) An apparatus for producing an audio signal as a replacement audio signal for a 
pre-recorded audio signal synchronized with a pre-recorded video signal, the apparatus 
comprising: 

one or more non-transitory computer readable mediums; 

a database populated with data representing the pre-recorded audio signal synchronized 
with the pre-recorded video signal, and data indicative of the timing of the pre-recorded audio 
signal relative to the pre-recorded video signal; 

an audio system for receiving and storing an input audio signal having timing similar to 
that of the pre-recorded audio signal; 

a comparing unit deriving from the pre-recorded audio signal first feature data encoding 
audible time-varying acoustic features of a predetermined type and the timing thereof in the pre- 
recorded audio signal and deriving from the input audio signal second feature data encoding 
audible time-varying acoustic features of the predetermined type and the timing thereof within 
the input audio signal, and determining timing differences existing between the first feature data 
and the second feature data; 

an editing unit for altering the timing of the acoustic features of the stored input audio 
signal to match the timing of the corresponding acoustic features of the pre-recorded audio signal 
to produce thereby an edited replacement audio signal; and 
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a timer for selecting a start time for starting outputting of the edited replacement audio 
signal to coincide with a corresponding start time of the pre-recorded audio signal relative to the 
pre-recorded video signal and for starting muting of the pre-recorded audio signal, 

wherein the database, the audio system, the comparing unit, the editing unit, and the timer 
are each stored in at least one of the one or more non-transitory computer readable mediums. 

41 . (New) An apparatus for producing an audio signal as a replacement audio signal for a 
pre-recorded audio signal synchronized with a pre-recorded video signal, the apparatus 
comprising: 

one or more non-transitory computer readable mediums; 

a database populated with data representing the pre-recorded audio signal synchronized 
with the pre-recorded video signal, and data indicative of the timing of the pre-recorded audio 
signal relative to the pre-recorded video signal; 

an audio system for receiving and storing an input audio signal having timing similar to 
that of the pre-recorded audio signal; 

a first acoustic feature extracting unit for deriving from the pre-recorded audio signal first 
feature data encoding audible time -varying acoustic features of a predetermined type and the 
timing thereof in the pre-recorded audio signal; 

a second acoustic feature extracting unit for deriving from the input audio signal second 
feature data encoding audible time -varying acoustic features of the predetermined type and the 
timing thereof within the input audio signal; 

a comparator for determining timing differences existing between the first feature data 
and the second feature data; 

an editing unit for altering the timing of the acoustic features of the stored input audio 
signal to match the timing of the corresponding acoustic features of the pre-recorded audio signal 
to produce thereby an edited replacement audio signal; and 

a timer for selecting a start time for starting outputting of the edited replacement audio 
signal to coincide with a corresponding start time of the pre-recorded audio signal relative to the 
pre-recorded video signal and for starting muting of the pre-recorded audio signal, 
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wherein the database, the database, the audio system, the first acoustic feature extracting 
unit, the second acoustic feature extracting unit, the comparator, the editing unit, and the timer 
are each stored in at least one of the one or more non-transitory computer readable mediums. 

42. (New) A method of producing an audio signal as a replacement for a pre-recorded 

audio signal synchronized with a pre-recorded video signal, the method comprising the steps of: 

populating a database in a non-transitory computer readable medium with data 
representing the pre-recorded audio signal synchronized with the pre-recorded video signal, and 
data indicative of the timing of the pre-recorded audio signal relative to the pre-recorded video 
signal; 

receiving and storing in a non-transitory computer readable medium an input audio 
signal having timing similar to that of the pre-recorded audio signal; 

providing first feature data that is derived from the pre-recorded audio signal and encodes 
audible time-varying acoustic features of a predetermined type and the timing thereof within the 
pre-recorded audio signal, 

providing second feature data that is derived from the input audio signal and encodes 
audible time -varying acoustic features of the predetermined type and the timing thereof within 
the input audio signal; 

determining timing differences existing between the first feature data and the second 
feature data; 

altering acoustic features of the stored input audio signal to produce thereby an edited 
replacement audio signal such that the timing of the acoustic features of the predetermined type 
of the edited replacement audio signal matches the timing of the acoustic features of the pre- 
recorded audio signal; and 

selecting a start time for starting outputting of the edited replacement audio signal to 
coincide with a corresponding start time of the pre-recorded audio signal relative to the pre- 
recorded video signal and for starting muting of the pre-recorded audio signal. 
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43. (New) An apparatus for producing an audio signal as a replacement audio signal for a 
pre-recorded audio signal synchronized with a pre-recorded video signal, the apparatus 
comprising: 

one or more non-transitory computer readable mediums; 

a database populated with data representing the pre-recorded audio signal synchronized 
with the pre-recorded video signal, and data indicative of the timing of the pre-recorded audio 
signal relative to the pre-recorded video signal; 

an audio system for receiving and storing an input audio signal having timing similar to 
that of the pre-recorded audio signal; 

a comparing unit deriving from the prc-rccordcd audio signal first feature data encoding 
audible time -varying acoustic features of a predetermined type and the timing thereof in the pre- 
recorded audio signal and deriving from the input audio signal second feature data encoding 
audible time-varying acoustic features of the predetermined type and the timing thereof within 
the input audio signal, and determining timing differences existing between the first feature data 
and the second feature data; 

an editing unit for altering the acoustic features of the stored input audio signal to 
produce thereby an edited replacement audio signal having acoustic features of the 
predetermined type with timing matching the timing of the corresponding acoustic features of the 
pre-recorded audio signal; and 

a timer for selecting a start time for starting outputting of the edited replacement audio 
signal to coincide with a corresponding start time of the pre-recorded audio signal relative to the 
pre-recorded video signal and for starting muting of the pre-recorded audio signal, 

wherein the database, the audio system, the comparing unit, the editing unit, and the timer 
are each stored in at least one of the one or more non-transitory computer readable mediums. 

44. (New) An apparatus for producing an audio signal as a replacement audio signal for a 
pre-recorded audio signal synchronized with a pre-recorded video signal, the apparatus 
comprising: 

one or more non-transitory computer readable mediums; 
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a database populated with data representing the pre-recorded audio signal synchronized 
with the pre-recorded video signal, and data indicative of the timing of the pre-recorded audio 
signal relative to the pre-recorded video signal; 

an audio system for receiving and storing an input audio signal having timing similar to 
that of the pre-recorded audio signal; 

a first acoustic feature extracting unit for deriving from the pre-recorded audio signal first 
feature data encoding audible time-varying acoustic features of a predetermined type and the 
timing thereof in the pre-recorded audio signal; 

a second acoustic feature extracting unit for deriving from the input audio signal second 
feature data encoding audible time -varying acoustic features of the predetermined type and the 
timing thereof within the input audio signal; 

a comparator for determining timing differences existing between the first feature data 
and the second feature data; 

an editing unit for altering the acoustic features of the stored input audio signal to 
produce thereby an edited replacement audio signal having acoustic features of the 
predetermined type with timing matching the timing of the corresponding acoustic features of the 
pre-recorded audio signal; and 

a timer for selecting a start time for starting outputting of the edited replacement audio 
signal to coincide with a corresponding start time of the pre-recorded audio signal relative to the 
pre-recorded video signal and for starting muting of the pre-recorded audio signal, 
wherein the database, the database, the audio system, the first acoustic feature extracting unit, the 
second acoustic feature extracting unit, the comparator, the editing unit, and the timer are each 
stored in at least one of the one or more non- transitory computer readable mediums. 



